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ABSTRACT 

When subjects are given open-ended stimulus 
situations allowing for responses at all possible stages of a 
developmental theory, the stage scores of individuals for each 
situation can be treated as scores on polychotomous items. Extensions 
of the concepts of difficulty and discriminating power from the 
dithotomous case to this ordinal category situation allow 
determination of these parameters. Assuming the trait being measured 
is distributed normally, stage difficulties or boundaries are 
deviates corresponding to cumulative proportions of subjects at each 
stage. Item discriminating powers are determined by the one factor 
loadings of the inter- item correlation matrix. Maximum-likelihood 
estimates of a subject's trait level can be determined from the 
parameters of each situation presented and that subject's answer 
pattern of stage scores. Data gathered in interviews to determine the 
level of moral development according to Lawrence Kohlberg's six-stage 
theory takes the above form. Results indicate that for different 
dilemmas, certain stages of reasoning are more easily attained than 
others, and when particular moral issues are analyzed across 
situations, an "entry" issue for various stages is indicated 
corresponding to Kohlberg's theory. (Author/MLP) 
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Psychcmetric Analysis 
of Developmental Stage Data 

Marcus Lieberman 
Harvard University 
Graduate School of Education 

Introduction 

Traditionally, psychometric ians have analyzed tests composed of items or tasks 
to which responses may be classified as "pass" or ''fail." Most typically, these 
are attitude or achievement measures often mixltidimensional in nature. The past 
few years have seen efforts focused on a new probabilistic, "latent trait" model of 
subject and item parameters, rather thstn the "true-score" model. Lord (1952) dis- 
cusses item and subject parameter estimation and Birnbaum (in Lord and Novick, 
1968) considers a nxamber of mathematical models which depend on the distribution of 
the trait in the population of subjects. It is this second model which will be 
applied to stage-type data. 

Stage theorists postulate: a series of structured wholes, forming an invariant 
sequence where passage from an inferior to a superior stage is equivalent to an 
integration. Often, "stage specific" tasks are presented to a subject and if 
almost all of a subset are passed and few of a more complex set, the subject is 
identified at a specific stage. Being in a transitional state is manifested by 
passing soatne tasks at both of two adjacent stages. The alternative to "stage 
specific" tasks is the presentation to the subject of an open-ended task, sentence 
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or situation, to which responses may be classified as representative of a particu- 
lar stage or mixtiire of two adjacent stages. The pattern of responses across 
several tasks yields an overall assignment to stage or traiisition. 

This open-ended task, sentence or situation may be considered to be an item 
with a r€uige of scoring possibilities luther than the dichotcmous pass-fail 
classification. Logical extensions of techniques employed to analyze tests of 
dichotomous items may be applied this polychotomous case. First, task diffi- 
culties and discriminating powers are determined; then, given a subject's response 
pattern to a number of tasks, a value for his latent trait which maximizes the 
probability of producing that pattern is estimated. 

In the following sections the techniques for estimating item and subject para- 
meters are discussed. It may be useful at this point, however, to see how they are 
related in a dichotomously scored example. Consider Figure 1. 

The "characteristic curves" of three items sire displayed. .Item C is the most 
difficult of the three because it takes much more of the trait to even have a 
finite probability of answering the item correctly. Item B is the most discrimina- 
ting, for below a certain level of the trait one has zero probability of answering 
the item correctly and above that level one has near certainty of doing so. 
Finite probabilities of answering Items A and C correctly exist over a large range 
of the latent trait. 
The Item Parameters 

The estimate of difficulty in the dichotomcusly scored item, when the trait of 
the population answering the item is assumed normally distributed, is the normal 
deviate corresponding to the proportion answering the item correctly. In extending 
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this notion to tiie polychotomous situation, the difficulties, or boundaries, are 
the normal deviates corresponding to the cumulative proportions of subjects scored 
at each stage. The hierarchical nature of the steiges allows us to state that a 
subject at a given stage has passed through all the previous stages. 

In dichotomously scored items, discriminating power is estimated in a niimber 
of different ways. The point biserial item-total correlation is one common 
method, often approximated by taking tLe difference between the proportion of top 
scoring subjects (based on total score) answering an itom correctly and the pro- 
portion of bottom scoring subjects also answering that item correctly. A second 
method suggests using the first factor loadings of the assumed unifactoral matrix 
of inter-item correlations. In the dichotomous case, these correlations are 
tetrachoric coefficients. 

In the multi-category situation, one might be tempted to use product-^noment 
correlation coefficients, but examination of the boundaries of typical polychotomous. 
items shows them to be non-equidistant in natiure, and simple arithmetic procedures 
are thus inadequate. The correct coefficient is the polychoric correlation. 

Pearson and Pearson (1922) first discussed this coefficient but computing pro- 
cedures were highly complex. A maximum likelihood e^itimation was proposed by 
Tallis (1962) and a simplified form when he\iristic estimates of difficulties are 
accepted rather than estimated is derived by Lieberman (19^9 )• 

Items are taken in successive pairs and a correlation coefficient is ccmputed 
for each pair. After coefficients are produced for all possible pairs of items, 
they are grouped in the familiar matrix form. Then a factor analysis using a 
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single factor {an assumption wbich must be tested) produce a loading for each 

item which will serve as a measure of that item's discriminp.ting power o 
The Subject Parameter 

Once the difficulties (or boundaries) and discriminating power for each item 
are determined, a subject's answer pattei'n of stage scores across a set of itemf> 
will produce an estimate of that subject's latent trait. The method described 
below will produce, for a subject randcmly chosen from a population whose trait 
may be assumed normally distributed, the same trait value for any subset of items* 
However, the more stories emp"'.oyed, the smaller will be the standard error of 
■{ihat value. 

The probability of being scored at a particular stage for a particular item 
is a function of a subject's latent trait value, the discriminating power of the 
item and the boundaries about that stage. Since the items are assumed locally 
independent, the probability of an entire pattern is merely the product of the 
probabilities of achieving each item score. For various values of the latent 
trait, it is possible to calculate the probabilities of a given pattern occurring, 
and the value that maximizes this probability is the maximum likelihood estimate 
of the latent trait value of a subject exhibiting this pattern. The likelihood 
equations and complete technique are described in Samejima (1969). 
An Application: The Development of Moral Judgment 

Kohlberg (1969) has postulated six stages in the development of moral judgment. 
These ways of reasoning about moral dilemmas are briefly described in Table 1, 
Researchers present a set of situations or dilemmas to subjects and probe their 



ERIC 



5* 



responses so as to get a sufficient amoiint of data to categorize the level of 
reasoniDS according co one of the six stages. Originally there was a single 
stage score for each dilemi^na but more recently an issue scoring system has "been 
devised which allovs a scorer to read c subject's references to a single moral 
issue (e,g,y the value cf life, the importance of rules, etc,) across all 
dilemmas a Then stage scores on various issues are recorded and an overall stage 
is calcuJ.ateda In the early research a weighted average was computed across the 
story or issue scores to arrive at a global score. 
The Stories 

A sajaple dilemma is given in Figure 2 accompanied by possible pix j3 questions 
an interviewer might ask in order to more precisely determine the evel of 
reasoning the subject might use* 

Tab? a 2 gives the stage boiindaries and discriminating power for each of five 
dilenffliasa It may be noted that some stories are more difficiilt than others at 
cej tain stages. Figure 3 graphically illustrates these boundaries and the non- 
interval distances between the stages is quite evident. 

A dramatic example of how using more information from the data than aji 
arithmetic averaige requir'^^s is pictured in Table 3. Here, patterns across the 
five stories yielding the same arithmetic average give different estimates of the 
latent trait due to the varying difficulties and discriminating powers of the 
stories. 

The poly c hot anous analogy to the item characteristic cxirve described earlier 
is pictured in Figure 3. Instead of the probability that a subject will pass an 



ERIC 



6. 

itea^ the probability that a subject will cross the boiindary between two aidjacent 
stages is presented. Note that the low discriminating power produces curves 
which give finite probabilities of crossing each boundary for a wide rsuige of the 
latent trait » moral judgment level. 

The value of the latent trait, as it stands, has little conceptual meaning 
other than to place each subject in a normative position to the population, 
though this is the variable that would be used in any group comparison analyses. 
To relate the value of the latent trait to the stages, one finds the probability 
that a subject with given trait value would respond to a "pure** pattern, i.e., 
mil through 66666 across the stories. This procedure reveals the probability 
that a subject is at any one of the six stages. A graphical representation of 
this relationship is given in Figure 
The Issues 

The various moral issues raised in the dilemmas are given in Figiare 5« Issue 
parameters are listed in Table h and bovindaries are illustrated in Figure 6, Note 
the highly increased discriminating powers reflecting the consistent thinking of 
subjects on a single moral issue in varying dilemmas. This increase in discrimina- 
ting power also is revealed in the issue characteristic curves given in Figure 7 
and the stage probabilities in Figure 8. 
Conclusions 

A great deal of information is provided by the graded response model. Story 
or task differences in difficulty and discriminating power may be estimated. A 
continuous, normally distributed trait level can be determined with its standard 
error for group comparisons. Apart from these data, the method provides a 
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description of the subject's probability of being in each stage, given his response 
pattern to the stories. 

These probabilities can be used in a variety of ways. For example, it may be 
interesting to know which children in a given group are "nearly ready" to change to 
the next highest stage. This occxirs when the probabilities of a subject being in 
adjacent stages are nearly equal. This information will be valuable to experi- 
menters attempting to change the level of reasoning with an intervention program 
of discussions, readings, or experiences. 

There are many developmental concepts which are being viewed as sequences of 
stages where each additional stage attained implies not only a substitution or 
addition but an integration with earlier stages. Some of these are ego develop- 
ment, physiological development, etc. In each, situations are discussed, tasks 
attempted or observations made. The researcher than rates the subject at a 
particular sta^e of development for each task, situation or observation. Using 
the technique described in this paper, situation, task or observation parameters 
may be estimated and used to determine subjects* latent trait levels for contrasts 
among groups of varying characteristics. 
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©LASSIFICATION OF MORAL JUDGMENT INTO LEVELS AND STAGES OF 

DEVELOPMENT 

Lsvols Basis of Moral Judgment Stages of Development 



I Moral value resides in external, quasi- 
g^iysicfll happenings, in bad acts, or in 

^ QU8^«physical needs rather than in 
imofw and standards. 



H Moral value resides in performing good 
or right roles. In maintaining the con- 
V9ntiortal order and the expectancies 
of othafBL 



ill Moral value resides in coriformity by 
the self to shared or shareable stand- 
ordfli rights, or duties. 



Stage 1: Obedience and punishment 
orientation. Egocentric deference to 
superior power or prestige, or a trouble- 
avoiding seL Obiective responsibility. 

Stage 2: Naively egoistic orientation. 
Right action is that instrumentally satts- 
fymg the self's needs and occasionally 
others'. Awareness of relativism of 
value to each actor's needs and per- 
apective. Naive egalitarianism and ori- 
entation to exchange and reciprocity. 

Stage 3: Good-boy orientation. Orien- 
tation to approval and to pleasing and 
helping others. Conformity to stereo- 
typical images of majority or natural 
role behavior, and judgment by in- 
tentions. 

Stage 4: Authority and social-ordfr 
maintaining orientation. Orientation to 
"dtMng duty" and to showing respect 
for authority and maintaining the given 
aocia! order for its own sake. Regard 
for earned expectations of others. 

Stage 5: Contractual legalistic orienta- 
tion. Recognition of an arbitrary ele- 
ment or starting point in rules or ex- 
pectations for the sake of agreement 
Duty defined in terms of contract 
general avoidance of violation of the 
will or rights of others, and majority 
will end welfare. 

Stage 6: Conscience or principle orien- 
tation. Orientation not only to actually 
ordained social rules but to principles 
of choice involv^ing appeal to logical 
universality and consistency. Orienta- 
tion to conscience as a directing agent 
and to mutual respect and trust 
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Story III. 

In Europe, a woman was near death from a special kind of cancer. There 
was one drug that the doctors thought might save her. It was a form of 
radium that a druggist in the same town had recently discovered. The drug 
was expensive to metke, but the druggist was charging ten times what the drug 
cost him to make. He paid $200 for the radium and charged $2,000 for a 
smal 1. dose of the drug. The sick woman's husband, Heinz, went to everyone 
be knew to borrow the money, but he could only get together about $1,000 
which is half of what it cost. He told the druggist that his wife was 
dying, and asked him to sell it cheaper or let him pay later. But the 
druggist said, "No, I discovered the driog and I'm going to make money from 
it«" So Heinz got desperate and broke into the man's store to steal the 
drug for his wife- 



1. Should Heinz stesa the drug? Why? 

2. Which is worse, letting someone die or stealing? Why? 
2a. What does the value of life meaja to you, anyway? 

3. Is there a good reason for a husband to steal if he doesn't love his 
wife? 



h. Would it be as right to steal it for a straiiger as his wife? Why? 

5. Suppose he was stealing it for a pet he loved dearly. Would it be right 
to steaa for the pet? Why? 

6. Heinz steals the drug and is caught. Shoiild the Judge sentence him or 
should he let him go free? Why? 

7. The judge thinks of letting him go free. What would be his reasons for 
doing so? 

8. Thinking in terms of society, what would be the best reasons for the 
Jxxdge to give him sane sentence? 

9. Thinking in terms of society, what would be the best reasons for the 
judge to not give him some sentence? 



TABLE 2 
STORY PARAMETERS 



itory. 




I 


III 


IV 


VII 


VIII 


Disc Power 




.3167 


.4817 


.3370 


.4474 


.5769' 


Stage Bound. 


1 


-1.16577 


-1.15902 


-1.34213 


-1.15567 ^ 


-1.57095 




2 


-0.18472 


-0.43518 


-0.06652 


-0.58667 


-0.67087 




3 


0.50850 


0.39344 


0.64134 


0.49074 


0.27614 




4 


1.15297 


1.35264 


1.58417 


1.28556 


, 1.11006 




5 


2.93346 


3.33138 


2.06890. 


2.60977 


3.58486 
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TABLE 3 

EXA]«i?r^s OP TH2 sa:iih global score 
pRODucixG various mopal xjt>zi'3::? levels 



Global Score 



2^0 



Pattern 



I II' III VII VIII 



3 2 2 



2 3 2 



2 "2 .2 



M.J. Level 

-1.2103 . 

-0.87^2 

-0.869^ 



260 



3 2 3 



2 2 .3 3 



3 2 2 



-0.6950 

-0.5737 
-0.5616 



280 



2 3 3 

3 3 2 



3 
3 



3 
3 



-0.1612 
-0'.lil68 



' .1 
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Figure 6 
The Issues 

A . The Legal System 

Ajj Rules of Law (and rules of custom with quasi-legal force) , (norms) 
G Relations of Punishment and of Legal Judgment (relations) 

E Civil Rights ^ Basic Rights to Liberty and Equity Which Law Must Protect (value) 
I Public Welfare and Order (value) 
B The System of Conscience 

Bjjj Obligation and Blame, Modes of Obligation, Responsibility 

Blame where rules and values conflict (norms) 
Bj Gnil'C, fear, anxiety and moral character-maintenance as motivating 

choice (value) 
By Personal Ethical Theory 

C The Affectional System 

Cjj Role Stereotypes ajid norms of good family and friendship roles (norms) 
C Relations of affection and concern for' welfare (relations) 
K Love > intimacy and sex (value) 

D The Leadership and Power System 

Djj Civil Stereotypes and norms of good authority and good citizen- 

follower roles (norms) 
D Relations of Authority , Respect, and Leadership (relations) 
E Civil Rights (value) 
L • Public Welfare and Order (value) 

» 

F The Economic System 

Fyj Work Role Stereotypes and norms (norms) 

F Relations of Contract and Equity (relations) 

I Property (value) 

J Truth and Trust (value) 

a Life (value) 
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Issue Parameters 



Discriminating Powers 





A B_ 
I 


Bjij C 


D F 


Q 


H 


I 




.885 .857 


.920 ,92k 


.851* .850 


.915 


.73U 


.892 






Stage Boundaries 








A 


-1.69370 


-.91291 


-0.08095 


I.I+007U 




2.U86U3 




-1.79661 


-.2U663 


.klkl9 


1.21009 




2.O9U68 


^III 


-1.91*1*96 


-1.01380 


.3U327 


1.27971 




2.3371*2 


C 


-2.00315 


-.95035 


.lt876U 


1.33877 




2.72390 


D 


-1.93982 


-1.38537 


-.03832 


1.38537 




5.1*7119 


F 


-2.33377 


-.83232 


.1U799 


1.18683 




2.33377 


G 


-1.69370 


-.8U162 


.06U73 


26338 




2.72390 


H 


-2.OIU51 


-.66587 


.38087 


..itoiiU 




2.1*1*087 


I 


-2.U806U 


-.75132 


-.053U5 


1.53579 




2.71852 
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